Antalya 25.8 Backport of #92250 - Reduce INSERT/merges memory usage with wide parts for very wide tables by enabling adaptive write buffers - #2166

Workflow [PR], commit [1fedca7]

Couldn't run the tests locally because I'm testing with debug by default and the test has an explicit no-debug in it. Will rely on the CI. But the PR landed without any issues, so I don't expect anything to go wrong. Will self-verify once CI finishes

@blau-ai

CI triage — 4 real failures, 0 caused by this PR

Verdict: All failing checks are either pre-existing/environmental (Iceberg & Swarms regression suites) or a flaky AST-fuzzer finding unrelated to this change. This PR touches only the MergeTree wide-part write-buffer / encrypted-disk write path; none of the failing tests exercise that path. The core suites that do — Fast test, all Stateless (incl. this PR's new 03770_min_columns_to_activate_adaptive_write_buffer), Stateful, Integration (5/5), Stress, BuzzHouse, all Builds — are green. PR and FinishCIReport are aggregate gates, red only because of the checks below.

1. Regression iceberg_1 / iceberg_2 / swarms — pre-existing

Decisive log excerpts:

• iceberg_1: [ Fail ] /iceberg/iceberg engine/rest catalog/position delete reads and [ Error ] /iceberg/iceberg engine/rest catalog/sort key timezone
• iceberg_2: [ Fail ] /iceberg/.../glue catalog/position delete reads, .../feature/position delete smoke, .../feature/compaction smoke
• swarms: [ Fail ] /swarms/feature/node failure/check restart swarm node

Evidence it's not this PR: sibling PR #2168 (an unrelated backport — "Fix crash in StorageDistributed when parsing malformed shard directory names", no MergeTree write changes) has the identical three failures — Regression x86 iceberg_1 (1 module errored), iceberg_2 (1 module failed), swarms (1 module failed) — in the same run batch. These are Iceberg catalog / position-delete / compaction and swarm node-failure features; they read external Iceberg tables and test distributed swarm restarts, and never go through the wide-part adaptive write buffer this PR modifies.

Next step: safe to ignore for this PR — track separately as flaky/broken regression suites on antalya-25.8.

2. AST fuzzer (amd_debug) — flaky

Decisive excerpt: Logical error: 'Unexpected return type from materialize. Expected String. Got Nullable(...)'.

This is a type-deduction assertion inside FunctionMaterialize on a fuzzer-mutated query — pure query-execution, no disk write, nothing to do with write buffers. The AST fuzzer randomly mutates existing test queries, so it hits different latent issues each run: the same sibling PR #2168's AST fuzzer passed (OK). This matches the known upstream class ClickHouse#107951 ("AST-fuzzer-internal-path: assertions on type-invalid inputs the analyzer normally rejects").

Next step: re-run the AST fuzzer (amd_debug) job; it should go green. It is not a blocker for this backport.

Health check

The change is a clean backport of upstream ClickHouse#92250: it adds the min_columns_to_activate_adaptive_write_buffer MergeTree setting (default 500) plus a SettingsChangesHistory entry, threads use_adaptive_write_buffer / adaptive_write_buffer_initial_size through WriteBufferFromEncryptedFile (with the growth logic in nextImpl), and updates the two gtest call sites for the new constructor signature. All directly-relevant CI — builds, fast/stateless/stateful/integration/stress/BuzzHouse and the newly added stateless test — is passing, so the change looks safe to merge once the pre-existing regression suites are dispositioned. No code changes needed from my side.

_{— @blau-ai, evidence-first from CI (I can't build/run ClickHouse locally).}